1. Field of the Invention
This invention relates to high dynamic range (HDR) imaging, and in particular, it relates to removal of ghost artifact during HDR image creation.
2. Description of Related Art
High dynamic range (HDR) imaging is a technique used in image processing and digital photography to handle sources that have extremely large ranges of brightness (light intensity). For example, an outdoor scene in daylight may include blue sky and sunlit objects as well as objects in shadows; a night scene may include neon lights and brightly lit objects as well as poorly lit objects; an indoor scene may include bright windows as well as darker areas, etc. These scenes pose a challenge for imaging devices such as digital cameras; the dynamic range of the image sensor of currently available digital cameras often cannot adequately image such scenes. If the exposure level is adequate for capturing details of darker areas of the scene, the brighter areas will often be overexposed with details lost; conversely, if the exposure level is adequate for capturing details of brighter areas of the scene, the darker areas will often be underexposed with details lost.
HDR imaging techniques deal with this problem by taking a set of multiple images of the same scene at various exposure levels, and then digitally merging the multiple images to create an HDR image that contains information from the original multiple images, so that details in both brighter and darker areas are adequately expressed in the HDR image. Methods for creating an HDR image from a set of multiple images are generally known.
Bracketing is a technique in photography for taking multiple images of the same scene using different exposure levels or other different setting values such as focus, depth of field, etc. Some cameras can perform autobracketing, i.e., automatically changing the setting multiple times and take multiple images. Each of the multiple images is sometimes referred to as a bracket. Multiple images generated by exposure bracketing can be used to create HDR images.
During HDR image creation, ghosting artifacts can appear when object have moved, appeared or disappeared in between the shooting of the different images (brackets). For example, during the shooting of three brackets, if a person walks into the scene only in the third bracket, then the HDR image created from the three brackets will have a semi-transparent figure of the person over the scene (“ghost”). These objects are referred to as moving objects or ghost-inducing objects in this disclosure.
Methods have been proposed to identify such ghost-inducing objects within the multiple images, so that the images can be processed to reduce or eliminate ghosting effects in the resulting HDR image. Some of these techniques are described in a review paper, A. Srikantha and D. Sidibé, Ghost Detection and Removal for High Dynamic Range Images: Recent Advances, Signal Processing: image Communications, 27(6), pp. 650-662, 2012.
In some known ghost artifact detection method, one of the multiple brackets is selected as the reference image (also referred to as a background image), which is the image that by definition does not contain any ghost-inducing objects. Other brackets are compared to the reference image to determine whether and where ghost-inducing objects are present. An advantage of such a single reference approach is that it simplifies the ghost detection and removal problem, because when processing each non-reference bracket, only two images (the reference bracket and the non-reference bracket) need to be compared to detect non-consistent pixels. A disadvantage, however, is that some large areas of the reference image may be badly exposed (either over or under exposed) and, if used as reference, result in unreliable ghost detection, lost of information and poor quality of the resulting HDR image in the relevant areas.
Thus, a method has been proposed to divide each image into multiple tiles (all brackets are divided into the same tiles), and for each tile, one bracket is selected as the reference image, so that different brackets may be selected as the reference bracket for different tiles. The selected bracket for each tile is referred to as local reference or local background. One such method is described in Gökdeniz Karadag and Ahmet Oguz Akyüz, Color preserving HDR Fusion for Dynamic Scenes, Journal of WSCG 20(2): 155-160 (2012) (“Karadag and Akyuz 2012”). FIG. 2, which is adapted from FIG. 1 of the Karadag and Akyuz 2012 paper, shows an image divided into multiple tiles; tiles labeled “S” (near the upper right hand and lower right hand corners of the image) indicate tiles for which the short exposure bracket will be used as the reference image, and tiles labeled “L” (near the center of the image) indicate those for which the long exposure bracket will be used (this image set had a total of three brackets). For the other tiles, the middle bracket will be used as the reference but for simplicity these tiles are not labeled in FIG. 1. After the reference bracket is selected for each tile, ghost pixels are detected in the non-reference brackets.